Aggregating Frame-Level Information in the Spectral Domain With Self-Attention for Speaker Embedding

Most pooling methods in state-of-the-art speaker embedding networks are implemented the temporal domain. However, due to high non-stationarity feature maps produced from last frame-level layer, it is not advantageous use global statistics (e.g., means and standard deviations) of as aggregated embeddings. This motivates us explore stationary spectral representations perform aggregation In this paper, we propose attentive short-time (attentive STSP) a Fourier perspective exploit local stationarity maps. STSP, for each utterance, compute through weighted average windowed segments within spectrogram by attention weights aggregate their lowest components form embedding. Because most map energy concentrated low-frequency region domain, STSP facilitates information retaining low only. Attentive shown consistently outperform on VoxCeleb1, VOiCES19-eval, SRE16-eval, SRE18-CMN2-eval. observation suggests that applying segment-level leveraging can produce discriminative